I want to start a program as root when booting, and will add one of the following lines to rc.local:
sudo /path/my_prog
sudo sh -c "/path/my_prog"
su -c "/path/my_prog"
What is the difference between this three lines, and which is the correct one?
You don't need any of them; rc.local runs with root privileges. The correct answer is to simply run your command.
/path/my_prog
su makes sense if you are root and want to switch to a different account.
sudo makes sense if you are running on an unprivileged account, and have been granted the rights to switch to another account (often, but not always, root), usually with the requirement to be able to interactively supply your password (though this can be turned off if you really have to; obviously, you need to understand what you are doing before you mess with security-related stuff).
sh -c "command" is just an inefficient way to run command. Running a shell makes sense when you actually require shell features such as wildcard expansion, redirection, etc, or shell builtins like cd.
Ideally, you might want to make sure that my_prog runs on a dedicated unprivileged system account; then, the syntax would be
su otheraccount -c /path/my_prog
where obviously you need to create otheraccount and make sure it has the privileges that are required for performing this particular task.
su optionally lets you add a lone dash before the account name to have it run a login session; but for a service account, this probably does not make sense.
sudo /path/my_prog will execute /path/my_prog with sudo privileges.
sudo sh -c "/path/my_prog" will execute /path/my_prog (specified by the flag -c) with sh using sudo privileges.
su -c "/path/my_prog" will execute /path/my_prog (specified by the flag -c) with your current shell using sudo privileges.
The correct one to use depends on your use case, so actually it's up to you. IMHO, sudo foo and su -c foo are basically the same.